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Abstract 


Model-based  system  engineering  methodologies  advocate  using  system  models  as  the  main 
vehicle  in  system  engineering  processes^^.  In  this  methodology,  a  system  model  represents  the 
relationships  and  interaction  between  the  entities  being  modelled.  Figure  2  depicts  an 
example  of  such  abstraction  of  the  interaction  within  and  between  two  subsystems. 


(a)  The  first  component  network 


(b)  The  second  component  network 


Legend:  Filled  circles  represent  actors  or  agents 

Filled  diamonds  represents  use  cases  or  components 

Different  colours  are  used  to  distinguish  actors  (agents)  or  use  cases 

(components)  in  each  subsystem. 

Figure  2  A  sample  component  network  of  two  subsystems 


As  a  result  of  the  difficulty  in  understanding  complex  relationships  within  comprehensive 
systems  models,  there  is  a  need  for  a  systematic  approach  in  assessing  properties  of  such 

models^^. 


Estefan,  J.  (2008).  Survey  of  model-based  systems  engineering  (MBSE)  methodologies.  Pasadena,  California. 
USA,  Jet  Propulsion  Laboratory,  California  Institute  of  Technology 

13  Brooks,  R.  J.  and  A.  M.  Tobias  (1996).  Choosing  the  Best  Model:  Level  of  Detail,  Complexity,  &  Model 
Performance,  Mathematical  and  Computer  Modelling,  Volume  24,  Number  4,  August  1996  ,  ppl-14 
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Lacking  evaluation  mechanism  for  system  models  presents  three  major  problems: 

(1)  difficulty  in  understanding  fundamental  properties  of  the  model  which  are  often 
attributed  as  a  major  reason  for  failure  of  the  system; 

(2)  lack  of  a  systematic  and  efficient  mechanism  in  ensuring  consistency  of  the  model 
through  all  stages  of  process,  system,  and  product  development and 

(3)  difficulty  in  understanding  which  components  perform  critical  functions,  and  which 
components  serve  as  a  bridge  between  sub-systems. 

This  paper  presents  a  two-step  approach  in  assessing  properties  and  consistency  of  the  model. 
The  definitions  of  the  properties  and  consistency  are  briefly  discussed  below: 

•  Properties  are  defined  based  on  a  set  of  network  science  measures^^.  To  use  the 
network  science  measures,  the  relationships  between  entities  in  the  system  model  are 
represented  as  an  entity  network  (see  Figure  1  for  a  simple  example).  The  network 
measures  can  be  computed  and  the  results  of  the  computation  can  be  explained 
meaningfully  within  the  system  engineering  discipline. 

•  Consistency  refers  to  the  congruent  between  entities  or  artefacts  developed  in  the 
system  development  process.  These  measures  can  be  quantitative  or  qualitative. 

Jiang  et  aP^  have  shown  that,  in  the  context  of  software  development,  analysing  properties  of 
a  model  provides  meaningful  feedback  for  the  purpose  of  design  and  system  verification 
processes. 

The  proposed  approach  provides  a  practical  mechanism  for  analysing  properties  of  the 
system.  The  major  contribution  of  this  work  is  two  folds: 

(1)  properties  of  system  models  can  be  used  at  both  network  and  node  level,  containing 
critical  information  on  the  overall  entity  network,  and 

(2)  consistency-assessment  measures  provide  a  mechanism  to  verify  consistency  of  the 
system  model. 

The  implication  and  significance  of  using  properties  of  nodes  within  the  context  of  system 
engineering  are  also  discussed. 


Van  Der  Straeten,  R.,  T.  Mens,  et  al.  (2003).  Using  description  logic  to  maintain  consistency  between  UML 
models.  «UML»  2003-The  Unified  Modeling  Language.  Modeling  Languages  and  Applications:  326-340. 
1^  Wasserman,  S.  and  K.  Faust  (1995).  Social  Network  Analysis:  Methods  and  Applications.  Cambridge, 
University  of  Cambridge  Press 

16  Jiang,  L.,  K.  M.  Carley,  et  al.  (2012).  The  Impact  of  Component  Interconnections  On  Software  Quality:  A 
Network  Analysis  Approach.  The  2012  IEEE  International  Conference  on  Systems,  Man,  and  Cybernetics 
(IEEE  SMC  2012) 
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Introduction 

Model-based  system  engineering  (MBSE) 

♦♦♦  MBSE  is  the  formalized  application  of  modelling 
to  support  system  requirements,  design,  analysis, 
verification  and  validation  activities  in  the  system 
engineering  life  cycle. 

□  Requirements  Models  -  Requirement  Diagram 

□  Design  Models  -  Package  Diagram,  Sequence  Diagram, 
Activity  Diagram,  State  Machine  Diagram,  etc. 
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Introduction  (Cont'd) 

*X*  Problems 

□  Hard  to  ensure  the  consistency  between  the  desigjis  and 


evolutions  of  design. 

□  Hard  to  identify  the  critical  elements  (nodes)  in  the  system 
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Introduction 

Introduction  (ConCd) 

Compute  the 
consistency 

How  to  verify  and  evaluate  the  system  models 
remains  challenges  in  both  industry  and  academia 
□  Status  quo  in  industry  practices 

Identity  the 
properties 

□  Research  -  major  focus  on  mathematical  approaches 
>  model-checking  and  automated  theorem  proving 

Application  to 
the  system 
integration 

>  using  description  logic  to  maintain  consistency 

Lacking  evaluation  and/or  verification  mechanism 
for  system  models  presents  three  major  problems 

Conclusion 

1 )  lack  of  a  systematic  and  efficient  mechanism  in  ensuring 
consistency  of  the  model 

2)  difficulty  in  understanding  which  components  perform 
critical  fimctions 

3)  difficulty  in  understanding  fundamental  properties  of  the 
model 
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Introduction 

The  Proposed  Approach 

The  Proposed 
Approach 

♦♦♦  An  approach  is  proposed  for  verification  and 
evaluation  of  the  models. 

Compute  the 
consistency 

The  approach  include  two  parts: 

(1 )  Define  a  set  of  measures  to  compute  the  consistency 
between  the  models. 

Identify  the 
properties 

(2)  Using  several  network  measures  to  identify  the  properties 
of  the  elements  in  the  model. 

>  Compute  the  complexity  of  the  model 

Application  to 
the  system 
integration 

>  Compute  the  properties  of  the  elements  in  the 
model. 

Conclusion 
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Assumption  with  the  approach: 

□  The  system  design  following  the  system  engineering 
process  and  SysML  (or  UML)  are  used  in  the  design. 

□  The  relationships  between  the  requirements,  objects, 
components  or  package  of  the  system  in  the  system 
models  are  well-established. 

□  Targeting  on  the  project  covering  the  entire  system 
development  lifecycle. 

The  applicability  of  the  approach  to  the  system 
integration  is  briefly  discussed  at  the  end  of  the 
presentation. 


Conclusion 
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Part  1 :  Compute  The  Consistency 
Between  The  Models 

The  Proposed 

Step  1 :  Define  a  set  of  measures 

Approach 

□  The  measures  are  divided  into  following  classes 

j 

>  Quantity  metrics 

Compute  the 
consistency 

■  counts  of  the  design  entities  and  relationships. 

Identify  the 
properties 

>  Complexity  metrics 

■  measure  the  relations  between  design  entities  and  the 

structure  of  the  proposed  system  architecture. 

Application  to 

y  Quality  metrics 

■  measure  the  relationship  between  the  desired  and  the  actual 

the  system 

characteristics  of  the  architecture. 

integration 

Conclusion 
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Part  1 :  Compute  The  Consistency 
Between  The  Models  (Cont’d) 

The  Proposed 
Approach 

*X*  Examples  of  the  proposed  quantity  metrics  for 

evaluation  of  the  models 

j 

Compute  the 

□  Number  of  Diagrams 

y  Package  Diagrams,  Use  Case  Diagrams,  Sequence 

consistency 

Diagrams,  State  Diagrams,  Activity  Diagrams, 

Identify  the 

Requirements  Diagram,  Class  Diagram 
□  Number  of  entities 

properties 

y  Requirements,  Use  Cases,  Actors,  Activities,  Package 

Application  to 

□  Number  of  design  relationship  type 

the  system 

>  Links  between  entities.  Interactions,  Activity  Flows, 

integration 

State  Transitions. 

Conclusion 
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*X*  Examples  of  the  proposed  complexity  metrics 


OvercdlDesignComplexity  =  1  - 


No_DesignEntities 
No_Relationships  +  No_Actors 


UseCase_  Complexity  =  1  - 


No_UseCase 

No_Relationships  +  No_Actors 


Application  to 
the  system 
integration 

Conclusion 


Object  _  Ifderation  _  Complexity  = 


No_of_Object  NoofClasses 

N  o_of_Obj  e  ct_Interac  ti  on  N  o_of_C  1  as  s_As  s  o  c  i  ati  on 
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Between  The  Models  (Cont’d) 

Step  2:  Understand  the  links  and  traceability  between 
the  models 


*  Objects  map  to  objects 

*  Classes  map  to  classes, 

*  Messages  map  to  the  links 
between  objects  or  classes) 


State  Diagram 


SequenceDiagram 

Tir- 


Realised  )y 


compl^j3flt€nted  by 


ClassDiagram 


Aggregated  to 


Block 

Diagram 

UseC 

]!ases 

Package  Diagram 

ilities”  requir  ?€rits  maps  to 


Other  System 
Design 
Consideration  [ 


System  architecture 

Hardware  (such  as  safety  and  reliability) 
Network  (such  as,  cable,  hub,  structure,  security) 
Operation  ^y^Mlteayc^oftware 
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Step  3 :  Define  a  set  of  consistency  measures 


^  ^  No  Require metnsRealisedByUseCases 

DegreeOfConsistencyreq_useca* - 

NoRequirements 


DegreeOfConsistencyusecase_sqD  = 


NoUseCasesModelledBySequenceDiagram 

No_UseCases 


DegreeOfCo  nsistencyseqo.  Class  Diagram  — 

NoClasses  InClassDiagram  +  NoObjectsInObjectDiagram 
No_Classes  InSequenceDiagram  +  NoObjectsInSequenceDiagram 


DegreeOfConsistencyciasses.  methods  — 

^  No_UndefinedMethodsRfiferences  +  No_UnderfmedParametErReferences 

No  DefinedMethods  +  No  DefinedParameters 
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Introduction 

Between  The  Models  (Cont’d) 

The  Proposed  ( 

ZasG  Study  1 :  Compute  the  consistency  between  the  models 

Approach 

□  Data  Sources;  Student  Group’s  Design  Documents: 

j 

□  Information  about  the  students: 

Compute  the 
consistency 

> 

Year  3  students  from  computer  science,  math  and  other 
engineering  program. 

Identify  the 
properties 

> 

Students  are  involved  in  Group  Project  with  5  to  6  group 
members. 

> 

Intensive  one  term -long  project  supervised  by  lecturers 

Application  to 
the  system 
integration 

> 

The  project  is  about  developing  a  robot  that  can  detect  mines  in 
the  “battle  field” 

> 

Students  are  guided  through  the  entire  engineering  process  from 
requirements  gathering  to  the  final  deliverables 

Conclusion 

> 

Students  uses  various  engineering  process  models 

> 

SRS,  SDD,  SPMP  are  compulsory  dehverables  and  presented 
during  the  processes  of  the  project 
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Part  1 :  Compute  The  Consistency 
Between  The  Models  (Cont'd) 

Case  Study  1 :  Compute  the  consistency  between  the  models 

□  Data  Sources:  Student  Group’s  Design  Documents: 

□  Information  about  the  students: 

□  Results 


Group  1 
(2010) 

Group  15 
(2010) 

Group  5 
(2011) 

Group  4 
(2011) 

DegreeOfConsistency  (Requirements 
and  Usecases) 

0.72 

0.83 

0.88 

0.69 

DegreeOfConsistency  (Usecases  and 
SequenceDiagram) 

0.60 

0.58 

0.83 

0.80 

DegreeOfConsistency  ( 
SequenceDiagram  and  Class  Digram) 

0.93 

0.57 

0.79 

0.49 

Overall  Consistency 

0.40 

0.27 

0.58 

0.27 

lAverage 

0.78 

0.69 

0.84 

0.67 
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elements  in  the  models  (Conf  d) 

The  Proposed 
Approach 

Examples  of  network  measures  used 
□  Network  level 

Compute  the 
consistency 

>  Network  Size,  Link  count.  Density,  Isolate  count 
(Component  count ),  Clustering  coefficient 

□  Node  level 

Identify  the 
properties 

y  Degree  centrality.  Betweenness  centrality. 

Eigenvector  centrality.  Closeness  centrality 

Application  to 
the  system 
integration 

Network  analysis  techniques  used 

□  Visualisation 

□  Computation  analysis 

□  Statistical  analysis 

Conclusion 
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Case  study  2 

□  Analyse  the  critical  nodes  in  the  two  use  case  diagrams 
for  two  subsystems  for  an  industry  project. 
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Note:  Filled-circles  represent  actors  or  agents  and  fiUed-diamonds 
represents  use  casesUnclassified 
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Part  2:  Identify  the  properties  of  the 
elements  in  the  model  (Cont’d) 

Case  study  2 

□  Analyse  the  critical  nodes  in  the  two  use  case  diagrams 
for  two  subsystems  for  an  industry  project. 

□  Compute  the  measures 

□  Analysis  the  results 


Rank 


Total 

degree 

centrality 


Subl  A  2 


Subl  A  1 


Sub2  A  1 


Subl  A  3 


Subl_UC 

8 


Betweenness 

centrality 


Sub2  A  1 


Sub2  A  2 


Sub2  UC  5 


Subl  UC  8 


Sub2  UC  3 


Sub2  UC  7 


Closeness 

centrality 


Subl  A  2 


Subl  A  4 


Subl  A  1 


Sub2  A  1 


Sub2  A  2 


Subl  A  3 


Eigenvector 

centrality 


Sub2  UC  5 


Sub2  A  2 


Sub2  A  1 


Sub2  A  3 


Sub2  UC  9 


Sub2  UC  1 
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♦♦♦  Conclusion; 

□  MBSE  provides  a  practical  approach  to  develop  complex 
systems 

□  Models  produced  in  the  system  engineering  processes 
have  to  be  evaluated  or  assessed  to  ensure  that  the 
requirements  are  fully  implemented,  and  models  are 
consistent  throughout  the  entire  engineering  process. 

>  The  proposed  approach  is  the  first  step  toward  addressing 
the  issue 

>  More  research  is  required  to  address  other  burning  issues 

□  In  order  to  have  better  understanding  of  the  system, 
models  have  to  be  studied  from  holistic  level 

>  Networks  science  provides  good  tools  for  studying  the 
hohstic  view  of  the  system,  the  interconnections,  and  their 
changes/evolutions 
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